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FOREWORD 


This  is  the  first  study  of  a  series  planned  to  be 
conducted  as  a  part  of  the  CAMI  general  aviation 
(GA)  human  factors  research  program.  The  fol¬ 
lowing  mission  statement  guides  the  overall  effort: 

Conduct  applied  human  factors  research  in  the 
laboratory  and  in  the  field  on  carefully  selected  GA 
problems,  to  obtain  objective,  scientifically  de¬ 
rived  data  which  will  aid  in  identifying  affordable 
options  for  reducing  the  risk  exposure,  and  num¬ 
ber  of  incidents  and  accidents  in  the  general  avia¬ 
tion  community,  and  which  will  serve  to  enhance 
GA  pilot  performance  under  non- routine  flying 
conditions. 

The  CAMI  general  aviation  human  factors 
research  program  is  consistent  with  the  FAA  policy 
statement  on  general  aviation,  promulgated  by 
the  FAA  Administrator  in  1993,  and  the  goals  of 
the  Flight  Standards  General  Aviation  Action 
Plan,  distributed  in  1992.  Development  of  the 
program  was  coordinated  with  AFS-800,  AFS- 
200,  AIR-2,  ACE- 100  and  with  guidance  by  the 
General  Aviation  Coalition,  accident  prevention 
and  pilot  training  working  groups.  FAA  human 
factors  program  management  support  was  pro¬ 
vided  by  AAR- 100. 

The  CAMI  GA  human  factors  research  pro¬ 
gram  incorporates  both  near-term  and  far-term 
objectives.  The  primary  near-term  focus  of  the 
program,  stressed  by  the  General  Aviation  Coali¬ 
tion,  is  to  develop  approaches  to  current  general 


aviation  problems  so  that  payoffs  in  reduced  risk 
exposure,  accidents  and  incidents  can  be  realized 
relatively  soon.  The  long-term  focus  of  the  pro¬ 
gram  is  directed  toward  future  problem  solutions 
utilizing  advanced  technologies  that  require  longer 
development  times  and  more  substantial  funding 
commitments.  These  two  program  approaches 
are  non-redundant,  mutually  supportive,  and  pro¬ 
vide  for  timely  human  factors  research  on  general 
aviation  safety  and  pilot  performance  issues  with 
payoffs  distributed  over  time. 

This  report  resulted  from  a  FY’94-95  effort  to 
consider  GA  cockpit  innovations  that  were  af¬ 
fordable  and  would  promise  a  more-or-less  im¬ 
mediate  enhancement  of  GA  pilot  performance. 
Mr.  Thomas  C.  Accardi,  Director,  Flight  Stan¬ 
dards  Service,  AFS-1,  sponsored  the  study.  Mr. 
Robert  A.  Wright,  Manager,  General  Aviation  & 
Commercial  Division,  AFS-800  and  Mr.  Michael, 
L.  Henry,  AFS-801,  provided  project  oversight. 
Dr.  Thomas  McCloy  and  Mr.  Ronald  Simmons, 
Human  Factors  Division  (AAR- 100)  of  the  Office 
of  Aviation  Research,  provided  human  factors 
program  management. 

The  CAMI  General  Aviation  Research  Simula¬ 
tion  Facility,  Human  Factors  Research  Labora¬ 
tory  (HFRL),  was  used  in  conducting  the  study. 
Dr.  Robert  E.  Blanchard  is  HFRL  manager.  Dr. 
Dennis  Beringer  was  principal  investigator,  as¬ 
sisted  by  Mr.  Robert  Touchstone  and  Mr.  Howard 
Harris. 
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Use  of  Off-The-Shelf  PC-Based  Flight  Simulators 
FOR  Aviation  Human  Factors  Research 


INTRODUCTION 

Flight  simulation  for  training  has  been  available  in 
one  form  or  another  for  over  75  years  (Edv^in  Link’s 
trainer  in  1929;  a  simulation  in  France  in  1917). 
Some  of  the  early  simulators  were  really  more  on  the 
order  of  “trainers,”  representing  a  class  of  devices  and 
not  any  specific  aircraft.  This  was  certainly  true  of 
Link’s  first  device,  which  was  relatively  simple  and 
was  assembled  from  readily  available  components 
from  a  number  of  non-aviation  applications.  This 
simple  device  ultimately  gave  way  to  more  compli¬ 
cated  and  far  more  expensive  devices  as  the  training 
community  sought  to  approximate  more  closely  the 
flight  dynamics  and  mission  environments  for  spe¬ 
cific  aircraft  (Valverde,  1968;  Eichler,  1974). 

The  development  of  the  digital  computer  greatly 
accelerated  advances  in  this  field,  and  these  digital 
machines  now  offer  the  opportunity  for  a  greater 
dissemination  of  flight  simulation  than  ever  before 
realized.  Recent  developments  in  processor  speed, 
video  memory/bandwidth,  and  memory  speed/den- 
sity  as  well  as  small  hard-disk  drives  with  previously 
unheard  of  capacity  mean  that  flight  simulations  can 
now  be  run  on  personal  computers  at  reasonable 
update  rates  and  with  out- the- window  views  that 
provide  a  moderate  level  of  scenic  detail.  The  poten¬ 
tial  for  applications  in  both  research  and  training  is 
substantial  given  the  represented  reduction  in  acqui¬ 
sition  and  maintenance  costs  and  ease  of  use.  The 
comparatively  low  cost  of  such  systems  may  justify 
reexamination  of  many  of  the  previously  held  beliefs 
concerning  what  the  necessary  criteria  are  for  useful 
flight  simulation. 

Given  the  compromises  that  one  must  accept  when 
conducting  flight  simulation  on  a  personal  computer, 
it  is  of  definite  interest  to  the  simulation  community 
to  determine  empirically  how  some  of  the  specific 
tradeoffs  engendered  in  the  PC-based  devices  may 
affect  both  the  efficacy  of  training  and  the  applicabil¬ 
ity  of  research  results  to  the  real  world.  It  is,  in  a  very 


real  sense,  a  continuation  of  the  discussions  about 
fidelity  and  its  effect  on  both  simulator  task  perfor¬ 
mance  and  learning.  In  these  continuing  discussions, 
answers  are  still  sought  for: 

1)  how  much  fidelity  is  needed  for  effective  transfer 

of  skills  or  for  research  data  to  generalize  to  the 

operational  environment, 

2)  how  much  of  the  task  can  be  effectively  repre¬ 
sented  in  simulation,  and 

3)  how  much  we  should  pay  for  that  box  (Hopkins, 

1975). 

Although  these  are  not  new  questions,  the  econom¬ 
ics  of  simulation  have  changed  such  that  we  can  now 
get  more  simulation  for  less  capital  investment.  This 
requires  us  to  ask  the  question,  “If  we  can  get  more 
simulation  for  the  same  investment,  what  is  the  ‘more’ 
that  we  should  ask  for?”  Simply  getting  “more”  does 
not  mean  a  better  or  more  effective  simulation.  In 
most  instances,  however,  the  question  of  what  more  to 
ask  for  is  not  being  asked  because  the  low  software 
costs  and  the  proliferation  of  personal  computers 
make  flight  simulation  much  more  affordable  and 
available.  There  is  no  perceived  need  to  weigh  the 
features  against  the  associated  costs  at  this  level  of 
simulation.  The  result  is  that  low-fidelity  flight  simu¬ 
lation  is  being  used  by  more  people  in  more  places 
than  ever  before.  The  economic  balance  point  has 
shifted  because  of  the  positive  cost  curve  acceleration 
(Figure  1)  being  reduced  (cost:  time  2),  as  compared 
with  previous  simulation  costs  (cost:  time  1),  which 
theoretically  increases  the  expected  transfer  per  dollar 
for  the  lower-fidelity  systems.  This  constitutes  the 
difference  between  historic  discussions  of  flight  simu¬ 
lation  and  the  current  weighing  of  the  issues,  where 
cost  is  now  a  facilitating,  rather  than  a  prohibiting, 
factor.  Some  simulation  programs  cost  less  than  one- 
hour’s  rental  of  a  single-engined  aircraft.  The  degree 
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of  positive  transfer  available  for  such  simulations 
needs  to  be  assessed  to  determine  if  the  actual  cost  of 
transferring  a  given  unit  of  knowledge  is  less  in  these 
simulations  than  in  previously  available  ones.  Trans¬ 
fer  in  the  training  environment  can  be  seen,  to  some 
degree,  as  similar  to  generalizability  in  the  research 
environment.  Taylor  (1995)  reported  substantive 
positive  transfer  from  such  devices  to  the  aircraft  for 
a  number  of  maneuvers.  Other  recent  studies  indicate 
that  these  simulations  have  utility  for  research  as  well 
(Thornton,  Braun,  Bowers  and  Morgan,  1992; 
Beringer,  Allen,  Kozak  and  Young,  1993;  Hennessy, 
Wise,  Koonce,  Smolensky  and  Garland,  1995). 

SPECIFIC  APPLICATION:  A  RESEARCH 
SIMULATOR 

Over  the  past  30  years,  much  has  been  published 
about  flight  simulation  for  training.  Although  much 
less  has  been  generated  dealing  specifically  with  re¬ 
search  simulation,  the  balance  of  publications  has 


recently  shifted  in  that  direction.  Eichler  ( 1 974)  made 
reference  to  the  use  of  flight  simulators  for  training 
and  for  engineering  research  and  development,  but 
virtually  no  mention  of  behavioral  research  as  an 
application  area.  It  was  interesting  that  in  their  Ap¬ 
pendix,  “Examples  of  Simulators,”  Jones  and  Hennessy 
(1985)  devoted  five  pages  to  engineering  simulators, 
seven  to  training  simulators,  but  only  three  and  a  half 
to  research  simulators.  A  quick  look  at  the  state  of 
things  in  1993  presents  a  picture  of  considerable  use 
of  simulation  for  behavioral  research.  In  the  Proceed¬ 
ings  of  the  Human  Factors  and  Ergonomics  society 
annual  meeting  of  that  year,  27  presentations  refer¬ 
enced  flight  simulation.  Of  these,  90%  used  it  as  a 
behavioral  research  tool  and  10%  used  it  for  perform¬ 
ing  behaviorally  oriented  research  on  simulation. 

There  are  a  number  of  reasons  for  one  to  simulate 
flight  rather  than  engage  in  the  actual  activity.  Jones’ 
(1967)  listing  of  many  of  those  justifications  bears 
repetition  here: 


Figure  1 .  The  hypothetical  relationship  between  system  fidelity  and  system  cost  (from  Roscoe, 
1980;  Figure  17.1) 
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1)  Consequences  of  inadequate  performance.  This 
can  he  reflected  in  economic  loss  or  physical  injury. 

2)  Cost  of  using  actual  system  for  training.  In  the 
case  of  commercial  airlines  use  of  an  aircraft  incurs 
revenue  plus  salaries. 

3)  Hazard  of  using  actual  system  for  training.  Avoids 
possible  undesirable  effects  on  society.  Can  simulate 
system  failures  (aircraft  crashes,  nuclear  power  plant 
fails). 

4)  Availability  of  actual  system  for  training.  Use 

where  system  is  not  available  for  training  (spacecraft) 
or  where  first  performance  in  new  system  must  be 
accurate  (single-place  aircraft). 

5)  Capability  of  actual  system  to  provide  emergency 
training.  Conditions  constituting  an  emergency  can¬ 
not  he  reproduced  in  system  without  great  expense  or 
damage  to  system. 

6)  Requirement  to  integrate  complex  team  activity. 

Integration  of  activities  of  large  number  of  people 
(complex  information  assembled  and  reacted  on 
quickly). 

7)  Susceptibility  of  task  to  stress.  Tasks  performed 
under  stressful  conditions  (space flight)  must  he  learned 
to  high  proficiency  level,  usually  difficult  on  actual 
system  due  to  impractical  nature  of  repeating  critical 
segments. 

8)  Capability  to  centralize  training  location.  Re¬ 
duced  number  of  training  locations  using  simulator 
may  reduce  cost  and  achieve  standardization. 

9)  Face  validity  of  training.  ^Realism*'  of  simulator 
may  be  necessary  to  assure  trainee  participation. 

1 0)  National  need.  Need  for  specialized  skills  may  be 
important  enough  to  prompt federal funding  for  simu¬ 
lation  equipment  to  speed  training  process. 

Although  this  listing  primarily  addresses  training 
issues,  some  are  applicable  to  research.  Both  cost  and 
hazard  are  issues  that  clearly  favor  the  use  of  simula¬ 
tion  for  flight  research.  The  face  validity  issue  may 
also  be  a  factor  in  the  attainment  of  generalizable 
research  results,  inasmuch  as  failure  of  the  participant 
to  ego  invest  in  the  simulated  process  is  likely  to 
negatively  bias  the  results;  responses  may  not  be  valid 
if  the  participant  does  not  have  to  “live”  with  the 
consequences  of  actions  and/or  decisions.  A  high 
motivation  to  “succeed”  among  pilots,  even  when 


flying  simulators,  may  have  the  positive  effect  of 
reducing  the  likelihood  of  unmotivated  performance, 
particularly  when  most  such  simulations  appear  to  be 
intrinsically  motivating.  Thus,  this  motivational  fac¬ 
tor  may  compensate,  in  some  degree,  for  shortcom¬ 
ings  in  face  validity  if  task  validity  is  high. 

Jones  and  Henry  (1985)  point  out  that: 

Simulators  for  research  differ  from  simulators  for 
engineering  design  and  development  principally  in  that 
the  former  use  has  the  objective  of  gaining  generalizable 
knowledge  (i.e.,  discovering  fundamental  principles 
that  are  broadly  applicable),  while  the  latter  use  is  to 
obtain  specific  data  for  the  design  of  particular  items  of 
equipment,  systems,  or  procedures  ...  Research  simula¬ 
tors  generally  must  allow  for  greater  latitude  in  manipu¬ 
lating  conditions,  events,  and  fundamental  functional 
characteristics  than  engineering  simulators. 

This  is  particularly  evident  when  one  contrasts,  for 
example,  the  simulation  platform  needed  to  develop, 
evaluate,  and  support  a  specific  airframe  product 
(Boeing  767,  for  example)with  that  designed  for  evalu¬ 
ating  different  types  of  manual  controls  and  control 
systems  or  for  examining  pilot  response  to  different 
cockpit  instrumentation  and  display  formats.  The 
differences  between  these  two  types  of  application 
platform  appear,  however,  to  be  fading  in  many  cases. 
One  of  the  major  concerns  in  research  simulations 
is  the  extent  of  control  available,  particularly  over 
ambient  conditions  not  controllable  in  the  real  world. 
It  should  be  recalled  that  most  behavioral  research  is 
reductionist  in  nature,  rather  than  holistic  (Egon 
Brunswik):  attempting  to  control  all  variables  with 
the  exception  of  those  being  studied.  It  can  be  argued 
that  reductionist  experiments  may  not  require  a  high 
degree  and  fidelity  of  flight  simulation  to  examine 
many  of  the  paradigms  of  concern  given  that  much  of 
the  experimental  space  is  represented  in  a  steady  state. 
That  degree,  the  extent  to  which  the  full  range  of 
operational  modes  are  represented,  can  be  curtailed  is 
clear,  specifically  when  only  a  particular  aspect  of 
operation  (e.g.,  landing)  is  under  examination.  The 
case  of  fidelity  (the  accuracy  with  which  the  system 
reproduces  operational  behavior)  may  not  be  as 
straightforward.  The  goal  is  to  generalize  from  ob¬ 
served  behavior;  the  extent  to  which  that  is  possible  is 
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a  function  of  the  extent  to  which  the  operator  behaves, 
in  simulation,  similarly  to  what  can  be  observed  in  the 
operational  environment.  Evidence  suggests  that  task 
procedural  fidelity  is  very  important  for  facilitating 
the  desired  behaviors.  Events  should  occur  in  the 
expected  sequence  with  the  expected  consequences. 

In  contrast  with  procedural  aspects  of  the  task, 
some  of  the  psychomotor  aspects  may  be  compro¬ 
mises  and  still  allow  adequate  performance  by  the 
pilot.  Force  characteristics  and  gain  in  manual  con¬ 
trols,  for  example,  might  be  of  a  lesser  fidelity  (given 
the  flexibility  of  the  human  operator)  so  long  as  they 
provided  adequate  input  to  the  system  for  achieving 
the  specified  tasks.  Common  sense  can  generally  dic¬ 
tate  many  fidelity  decisions  with  some  notable  excep¬ 
tions.  It  should  be  clear  that  the  study  of  navigation 
computer  interfacing  with  the  pilot,  involving  graphi¬ 
cal  interface  and  menu  hierarchy  design,  will  not 
likely  require  the  fidelity  of  manual  flight  control 
input/output  that  one  would  need  to  execute  manual 
control  theory  studies.  This  selective  use  of  high 
fidelity  has  been  the  historical  exception  rather  than 
the  rule,  with  many  simulation  efforts  striving  to 
achieve  high  fidelity  across  the  board.  Such  an  ap¬ 
proach  generally  increases  both  the  acquisition  and 
maintenance  costs  of  a  system. 

Previous  barriers  to  the  general  use  of  flight  simu¬ 
lations  in  behavioral  research  largely  involved  these 
costs  of  acquisition  and  maintenance  and  system 
availability  because  many  of  the  simulators  had  been 
developed  for  training  or  as  engineering  simulators 
within  a  full-mission  context.  The  research  commu¬ 
nity  now  stands  to  benefit  from  the  development  of 
lower-cost  simulations  that  can  reasonably  represent 
selected  flight  tasks  of  interest.  Such  simulations  have 
the  promise  of  allowing  research  to  be  conducted,  for 
a  greatly  reduced  investment  {cost factor),  that  will  still 
generalize  to  the  operational  environment.  The  op¬ 
portunity  then  is  also  present  to  use  the  lower-cost 
lower-fidelity  devices  to  perform  screening  experi¬ 
ments,  reducing  the  resources  required  from  higher- 
fidelity  simulations  {availability  factor)  and  freeing 
them  for  more  focused  investigative  efforts.  It  should 
also  be  noted  that  simpler  systems  tend  to  have  higher 
reliabilities  as  well  as  lower  maintenance  costs. 


Hardware  &  Software  Components 

Specific  constraints  imposed  on  the  present  system 
development  were  that  it  (1)  had  to  be  operational 
within  6  months  of  project  initiation,  (2)  had  to 
reasonably  represent  a  familiar  (popular)  single-en¬ 
gine  general-aviation  aircraft  and  its  environment 
(instrumentation,  controls,  and  external  visual  cues), 
and  (3)  needed  to  meet  the  criteria  of  a  “research” 
simulator,  to  the  extent  that  it  allowed  manipulation 
of  experimental  dependent  variables  of  present  inter¬ 
est  and  the  extraction  of  relevant  performance  data.  It 
was  with  these  goals  in  mind  that  the  moderate- 
fidelity  general  aviation  flight  simulation  at  CAMI, 
referred  to  as  the  Basic  General  Aviation  Research 
Simulator  (BGARS)  (Beringer,  1994),  was  developed 
using  widely  available  80486-based  personal  comput¬ 
ers  (50  &  66  MHz)  to  generate  a  comparatively  rich 
simulated  flight  environment.  The  criteria  of  low  cost 
and  short  development  time  argued  in  favor  of  using 
commercial  off-the-shelf  products  as  much  as  pos¬ 
sible.  Thus,  all  hardware  and  software  selected  were 
available  as  commercial  products.  Minor  modifica¬ 
tions  were  performed  in  cases  where  components 
needed  to  communicate  but  did  not  already  embody 
the  necessary  features. 

The  system  includes  variable  flight  instrumenta¬ 
tion,  forward,  45-  and  90-degree  left  external-world 
views,  and  a  map  display.  Left-side  external  views  were 
included  to  facilitate  VFR  flight  and  the  execution  of 
visually  referenced  left-hand  traffic  patterns.  A  sim¬ 
plified  block  functional  diagram  of  the  system  appears 
in  Figure  2.  High-fidelity  analog  control  inputs  are 
provided  (e.g.,  damped/self-centeringyoke,  high-per¬ 
formance  throttle  quadrant,  gear,  flap,  and  trim  con¬ 
trols;  radio  controls).  The  simulation  is  based  on 
commercially  available  flight  simulation  programs, 
one  (FS-lOO)  an  instrument  flight  trainer  and  the 
other  (ATP),  a  “game”-type  flight  simulation.  The 
modified  FS-lOO  package  provides  the  cockpit  dis¬ 
plays,  control  input  processing,  continuous  collection 
of  16  performance  variables,  choice  of  either  a  Beech 
Bonanza  (A-36)  or  Beech  Sundowner  aero  model,  and 
concurrently  feeds  six-degree-of-freedom  data  to  the 
second  program,  producing  the  out- the-window  view. 
This  latter  program  is  used  to  produce  all  outside 
views,  one  per  processor/display  combination.  Thus,  the 
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three  outside  views  (Figure  2)  require  three  separate 
processors,  each  running  a  copy  of  the  same  program 
but  selected  to  depict  the  appropriate  viewing  vector. 
The  forward  view  is  projected  to  obtain  accommoda¬ 
tion  distances  exceeding  3  meters  and  a  55-degree 
field  of  view.  All  interprocessor  communications  are 
serial,  obviating  any  need  for  network  hardware  or 
software.  Although  the  initial  configuration  used  five 
processors,  at  an  overall  cost  of  approximately  $25,000 
(hardware/software),  an  acceptable  simulation  (i.e., 
forward  view  and  instruments)  can  be  produced  using 
only  two  computers;  each  added  function/view  re¬ 
quires  another  processor  (Figure  2).  A  more  detailed 
system  diagram  depicting  the  revised  system  using 


Pentium  100  MHz  processors  can  be  found  in  the 
Appendix.  Included  in  the  diagram  is  a  sixth  processor 
used  to  deliver  ATC  and  pseudo  pilot  verbal  messages 
from  digitized  sources. 

Software  Considerations  for  Integration 

The  simultaneous  use  of  two  previously  self-con¬ 
tained  simulation  programs,  each  with  its  own  inter¬ 
nal  representation  of  the  real  world,  required  that  a 
number  of  issues  be  addressed  before  an  acceptable 
system  could  be  fielded.  These  included  issues  related 
to  the  geographic  database,  communications  between 
programs,  and  the  specific  requirements  of  the  re¬ 
search  application. 


Figure  2.  Simplified  schematic  of  the  BGARS,  version  1 . 
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Geographic  location  of  airports.  Although  the  system 
intercommunications  considerations  were  relatively 
straightforward  and  elegant  in  their  simplicity,  the 
architecture  requires  that  each  processor/software 
package  in  the  system  contains  a  geographic  database 
identical  with  those  in  the  other  systems.  One  needs 
to  be  aware  that  not  all  databases  are  equal  and  most 
low-end  flight  simulations  distributed  for  the  mass 
market  use  the  National  Oceanic  &  Atmospheric 
Administration  (NOAA)  database;  it  is  readily  avail¬ 
able  for  minimal  cost  and  does  not  require  licensing 
for  use.  Some  of  the  more  expensive  instrument  train¬ 
ing  packages  use  the  Jeppeson-Sanderson  (J-S)  data¬ 
base.  This  commercial  product  is  expensive  and 
requires  licensing,  but  its  data  are  updated  very  fre¬ 
quently.  The  two  databases  are  not  congruent;  indeed 
we  have  found  displacements  orthogonal  to  runway 
centerlines  ranging  from  50  to  200  feet.  This  can  be 
most  unnerving  when  one  flies  a  “perfect”  Instru¬ 
ment-Landing-System  (ILS)  approach  and  finds,  on 
short  final,  that  the  aircraft  is  not  aligned  with  the 
runway.  We  have  obtained  editing  functions  for  the 
package  (ATP)  using  the  NOAA  database,  allowing  us 
to  move  the  airports  into  alignment  with  the  J-S  data¬ 
base.  In  most  cases,  the  adjustment  required  is  small. 

A  related  concern  is  the  fidelity  with  which  airport 
facilities  are  represented.  We  regularly  found  discrep¬ 
ancies  between  the  representation  of  an  airport  in  the 
instrument-package  database  (and  thus  moving  map 
display)  and  the  depiction  in  the  out-the-window 
scene.  Secondary  runways  were  missing  from  the  out- 
the-window  view  in  a  number  of  instances.  This 
appeared  to  be  more  likely  at  “secondary”  airports  as 
primary  airports  generally  had  accurate  runway  and 
taxi  way  depictions.  It  is  not  clear  to  what  extent  these 
discrepancies  represent  changes  in  the  facilities  re¬ 
quiring  updating  of  the  database  and  to  what  extent 
they  are  simply  omissions.  Thus,  it  is  recommended 
that  one  not  choose  airports  as  primary  or  secondary 
destinations  (or  even  emergency  fields)  without  fully 
assessing  the  agreement  between  the  two  databases. 

Altitude.  Differences  in  field  elevation,  however, 
are  a  more  difficult  problem.  Each  software  package 
represents  altitude  in  a  different  fashion.  FS-lOO 
assumes  terrain  elevation  to  be  that  of  the  nearest 
airport.  ATP  uses  an  interpolative  approach  to  deter¬ 


mining  altitudes  at  locations  falling  between  major 
reference  points  to  approximate  intermediate  terrain 
elevations.  Add  to  this  small  differences  (50-100  feet) 
between  the  two  source  databases  and  the  result  is 
some  interesting  anomalies.  We  found  an  initial  dif¬ 
ference  between  databases  of  50  feet  at  Will  Rogers 
World  Airport  (OKC,  runway  35  right),  which  occa¬ 
sioned  a  rather  high  and  sudden  contact  with  the 
ground  on  our  first  simulated  approach  (it  was  liter¬ 
ally  a  highway  in  the  sky),  the  field  elevation  in  the 
visual  scene  database  being  lower  than  the  field  eleva¬ 
tion  in  the  instrument  package  database.  Although 
ATP  can  make  an  initial  adjustment  on  start-up  to  the 
current  field  elevation,  differences  at  the  destination 
of  a  flight  can  still  produce  the  aforementioned  effect. 
This  problem  was  resolved  by  transmitting  altitude 
above  ground  level  (AGL)  to  the  scene-generating 
packages.  Flying  over  mountains  represented  in  the 
visual  database  does  not  produce  unusual  effects, 
because  FS-IOO  is  not  aware  of  the  mountains  if  no 
airport  is  located  there,  and  ATP  uses  the  base  of  the 
slope  as  terrain  elevation. 

Heading.  One  other  concern  is  the  method  by 
which  heading,  also  represented  as  the  forward  view¬ 
ing  vector,  is  communicated  between  packages.  Recall 
that  heading  has  both  true  and  magnetic  representa¬ 
tions  and  that  both  simulation  packages  must  agree  on 
the  method  for  handling  and  communicating  those 
data.  An  early  experience  during  development  was, 
again,  a  distressing  one  on  approach  to  San  Francisco 
(SFO)  when  all  attempts  to  fly  straight  down  the 
runway  produced  extreme  drift  to  the  right.  Mis¬ 
matches  regarding  true  and  magnetic  headings  and  a 
difference  in  the  internal  representations  of  data  be¬ 
tween  the  two  software  packages  contributed  to  this 
effect.  Processing  of  transmitted  heading  was  modi¬ 
fied  to  correct  this  mismatch. 

Collision  with  ground  or  objects.  The  enabling  of 
collision  detection  has  two  levels.  The  first,  already 
mentioned,  requires  that  both  packages  operate  from 
the  same  basic  data.  Compatibility  is  critical  for  a 
multimachine  one-way  communication  environment 
because  no  handshaking  or  flow  control  is  being 
executed;  one  machine  is  the  transmitting  “host” 
while  the  others  are  passive  listeners  (peripherals). 
The  second  level,  object  collisions,  can  be  achieved  by 
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having  the  forward- view- generating  processor  deter¬ 
mine,  as  the  software  is  capable  of  doing,  if  an  object 
in  the  visual  scene  has  been  struck  and  then  send  a 
signal  back  to  the  host.  This  would  require  only  two 
processors  to  have  full  duplex  operation,  with  the 
other  machines  remaining  passive  monitors  of  one 
side  of  the  communications  flow. 

Visual  display  considerations  regarding  color,  bright¬ 
ness,  and  aliasing.  Not  all  software  packages  present 
visual  data  in  the  same  manner.  The  temptation  exists 
to  opt  for  the  highest-resolution  out- the- window 
imaging  available  in  order  to  reduce  aliasing  along  the 
horizon  line  and  on  any  other  features  with  linear 
boundaries.  FS5  (in  VGA  graphics  mode),  with  its 
textured  terrain  depiction,  does  exhibit  less  aliasing 
than  does  ATP  (in  EGA  graphics  mode)  as  a  function 
of  the  display  resolution  being  used.  However,  FS5 
tends  to  have  a  much  darker  overall  affect  with  its 
textures  than  does  ATP  with  its  uniform  area-fill  color 
(primarily  light  green  or  brown  with  occasional  yel¬ 
low  or  orange  blocks).  While  this  may  not  be  of  much 
concern  when  displayed  on  a  large  direct-view  CRT, 
the  brightness  issue  becomes  critical  when  a  projec¬ 
tion  system  is  used.  The  image  produced  by  the  GE 
projection  CRT  was  of  an  acceptable  level  of  bright¬ 
ness  when  the  backgrounds  were  area  filled  and  prima¬ 
rily  light  green  or  light  gray  (urban) ,  but  the  brightness 
was  not  acceptable  in  the  initial  adaptation  of  FS5  and 
terrain  features  and  runways  were  difficult  to  detect. 
A  subsequent  modification  was  made  to  brighten  all 
of  the  colors,  but  the  effect  was  to  wash  out  much  of 
the  contrast  and  lose  some  of  the  definition.  Some 
additional  tuning  of  brightness  and  contrast  may 
resolve  this  problem.  Detection  of  runways  and  run¬ 
way  centerlines  was  also  more  difficult  at  a  distance 
with  the  textured-terrain  display  provided  by  FS5 
than  with  the  simpler  area-fill  graphics  of  ATP.  This 
may  or  may  not  be  desired,  depending  upon  the  tasks 
to  be  performed. 

Required  modifications  for  research.  Two  modifica¬ 
tions  were  required  to  make  the  system  suitable  for 
research.  These  were  to  provide  (1)  the  ability  to 
manipulate  the  independent  variables  of  interest  and 
(2)  the  ability  to  record  dependent  variables  of  inter¬ 
est.  The  first  two  studies  required  selectable  instru¬ 
mentation,  as  we  were  interested  in  piloting 


performance  with  both  conventional  navigation  dis¬ 
plays,  as  represented  by  the  very-high-frequency  omni 
range  (VOR)  indicator  and  directional  gyro  (DG), 
and  integrated  displays,  as  represented  by  the  horizon¬ 
tal  situation  indicator  (HSI).  Additionally,  we  were 
interested  in  the  effects  of  simple  memory  aids/instru¬ 
ment  bugs.  The  HSI  was  already  available  in  the 
software;  the  modification  allowed  selection/deletion 
of  bugs  on  the  DG  and  altimeter.  Thus,  requirements 
of  (1)  were  satisfied.  Meeting  the  requirements  of  (2) 
was  facilitated  by  the  fact  that  most  of  the  dependent 
variables  ofinterest  were  already  being  recorded  in  the 
replay  memory.  This  required  only  the  addition  of  a 
header  record  to  identify  the  data  file,  a  sample  num¬ 
ber  for  each  data  slice,  an  event  marker  that  could  be 
inserted  by  the  experimenter  in  real  time  from  the 
keyboard,  and  lateral  error,  in  feet,  from  the  VOR/ 
localizer  course  (for  a  total  of  16  variables).  An  addi¬ 
tional  switch  was  added  to  allow  data  collection  to  be 
paused  during  a  run  while  the  aircraft  continued  to 
fly.  A  package  was  also  provided  that  allows  data  files 
to  be  accessed  and  rewritten  in  ASCII  format  for 
subsequent  use  by  data  reduction  and  analysis  packages. 

Flight  task  difficulty /aero  models.  The  original  aero 
model  provided  was  a  Beech  Bonanza  A-36.  This  was 
used  in  the  initial  evaluation  with  instructor  pilots 
and  was  found  to  be  too  sensitive  to  control  inputs 
and  somewhat  unstable  longitudinally.  It  did,  how¬ 
ever,  provide  a  significant  challenge  to  the  participat¬ 
ing  pilots,  and  thus,  a  good  task  loading  that  was  likely 
to  exercise  the  pilot  and  system  in  such  a  way  as  to 
expose  effects  due  to  the  experimental  manipulations 
being  used.  A  simplex  (single-engine,  fixed-pitch  prop, 
fixed  gear)  model  was  desired  for  use  with  the  private 
pilot  samples  to  be  examined  and  a  Beech  Sundowner 
model  was  added.  Its  flying  characteristics  were  quite 
similar  to  the  actual  aircraft,  and  pilots  without  complex 
aircraft  experience  found  it  comparatively  easy  to  fly. 

Update  rates  and  throughput.  The  initial  installa¬ 
tion  of  the  system  used  80486  processors  for  the  flight 
instruments/aero  model  package  (66  MHz)  and  out- 
the- window  views  (33  MHz).  Different  update  rates 
were  obtained  for  the  instrument  package  and  the 
external  views.  The  instrument  panel  updates  were 
generally  on  the  order  of  12  to  16  Hz.  The  out-the- 
window  views  involved  much  more  in  the  way  of 
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graphics  and  generally  did  not  operate  much  faster 
than  6  to  10  Hz.  This  update  rate  was  not  objection¬ 
able  for  most  operations  and  only  became  noticeable 
in  steep-banked  turns.  The  upgrading  of  the  system  to 
100  MHz  Pentium  processors  with  the  PCI  bus  for  all 
computers  excepting  the  map  display  (486  66MHz) 
after  conclusion  of  this  study  significantly  increased 
throughput.  Instrument  and  all  out- the-window  view 
update  rates  appear  to  be  at  or  above  16  Hz.  Some 
occasional  stepping  (discrete  observable  movements) 
can  be  observed  in  instrument  indications,  particu¬ 
larly  in  the  GDI  needle,  but  this  appears  to  be  a 
function  of  calculational  and  display-resolution  arti¬ 
facts  and  not  system  throughput. 

Advantages 

The  advantages  of  this  approach  are  (1)  low  cost  of 
hardware,  often  already  available  on  site,  (2)  compara¬ 
tively  low  cost  of  software,  as  modifications  were 
minor,  as  compared  with  development  of  a  com¬ 
pletely  customized  system,  (3)  modularity  of  both 
hardware  and  software,  allowing  upgrade  of  any  of  the 
components  or  easy  expansion  of  simulation  by  add¬ 
ing  components,  and  (4)  simplicity  of  the  communi¬ 
cations  protocol.  The  low  cost  and  ease  of  assembly/ 
integration  allow  multiple  “standardized”  systems  to 
be  distributed  for  cooperative  inter-laboratory  stud¬ 
ies.  This  approach  appears  to  have  great  utility  for 
both  research  and  training. 

Disadvantages 

One  should  expect  that  the  advantages  noted  do 
not  come  without  cost.  The  use  of  commercial  off- 
the-shelf  software  without  access  to  the  source  code 
poses  a  potential  problem  for  investigators  who  wish 
to  manipulate  variables  not  directly  accessible  through 
the  program.  Although  we  have  had  reasonable  suc¬ 
cess  in  working  with  the  developers  of  the  software  to 
obtain  modifications  necessary  to  make  the  simula¬ 
tion  useful  for  research,  there  are  yet  some  areas  where 
desired  modifications  are  not  immediately  obtain¬ 
able.  This  is  due  either  to  scheduling  conflicts  in 
securing  development  time  or  to  changes  that  have 
major  impact  on  the  structure  of  the  software.  One 
solution  to  the  problem  that  we  have  pursued  is  the 


addition  of  processors  on  the  serial  distribution  to 
provide  additional  features  (i.e.,  ATC  and  pseudo 
pilot  digitized  and  automated  voice  inputs).  The 
researcher  then  maintains  control  of  the  auxiliary 
functions  and  can  modify  and  develop  the  code  as 
needed.  This  approach  can  also  be  used  to  develop 
auxiliary  instrument  displays,  given  that  the  requisite 
data  are  available  on  some  communication  line  com¬ 
ing  from  the  host  processor.  Additional  modifications 
are  being  made  to  the  software  that  will  provide  access 
to  more  data  variables  in  real  time  as  well  as  to  some 
discrete  failure  modes,  multiplying  the  options  avail¬ 
able  to  the  experimenter. 

EMPIRICAL  VALIDATION 

Having  now  fielded  a  functioning  system,  it  was 
necessary  to  validate  the  utility  of  this  system  for 
research.  A  problem  area  was  selected  during  develop¬ 
ment,  as  previously  mentioned,  that  would  allow 
experimental  outcomes  to  be  compared  with  out¬ 
comes  of  other  aircraft-  and  simulator-based  research 
for  at  least  a  preliminary  empirical  validation  of  the 
BGARS  as  a  research  tool. 

Problem 

The  selected  comparisons  were  (1)  between  alter¬ 
nate  ways  of  presenting  course  deviation  information 
for  very-high-frequency  omni  range  (VOR)  naviga¬ 
tion  and  (2)  between  instrument  formats  both  with 
and  without  short-term  memory  aids.  The  Horizontal 
Situation  Indicator  (HSI)  has  seen  considerable  use 
and  combines  the  functions  of  the  very-high-fre¬ 
quency  omni  range  (VOR)  and  directional  gyro  (DG) 
indicators  within  a  single  instrument  (a  design  sug¬ 
gested  by  Walter  Grether;  Williams,  1949,  as  re¬ 
printed  in  Roscoe,  1971).  There  has  been  little  doubt 
that  the  HSI  simplifies  the  pilot’s  task  of  integrating 
the  various  pieces  of  data,  with  some  attendant  gains 
in  the  performance  of  tracking  and  orientation  tasks. 
Short-term  memory  aids  or  “bugs”  are  available  to 
relieve  the  pilot  of  the  tasks  of  recalling  target  altitudes 
and  headings.  It  was  anticipated  that  the  aided  (HSI 
and  bugs)  conditions  would  produce  performance 
superior  to  that  obtained  in  the  unaided  (VOR/DG 
and  no  bugs)  conditions. 
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The  question  at  hand  was  one  of  cost  effectiveness: 
Did  performance  enhancements  associated  with  the 
HSI  and  bugs  justify  the  expense  of  acquiring  and 
installing  such  devices  in  comparatively  inexpensive 
general  aviation  aircraft?  The  questions  relative  to  the 
simulation  were  several,  involving  (1)  sufficient  task 
fidelity  to  motivate  generalizable  behavior,  (2)  ability 
of  the  system  to  collect  adequate  continuous,  real¬ 
time  performance  data,  and  (3)  stimulation  of  the 
same  types  of  procedural  errors  as  those  seen  in  the 
operational  environment  through  appropriate  task 
and  workload  representation.  The  reliability/ avail¬ 
ability  of  such  a  system  was  also  of  interest  as  com¬ 
pared  with  other  such  devices.  A  brief  summary  will  be 
presented  to  address  these  questions. 

Task  Fidelity  and  Generalizability 

More  than  36  pilots  flew  the  simulation  over  the 
course  of  three  phases  of  the  initial  study.  Of  these 
individuals,  12  were  experienced  pilots  (with  more 
than  500  hours,  of  which  at  least  30  hours  were  logged 
in  the  last  6  months),  most  of  whom  were  instructors, 
with  the  remainder  being  private  pilots  (with  less  than 
200  hours  of  flight  time  and  less  than  20  hours  in  the 
last  6  months).  Each  participant  flew  the  simulator  for 
two  2-hour  sessions,  the  first  session  for  familiariza¬ 
tion  and  training  and  the  second  for  data  collection. 
Training  included  traffic  patterns,  constant-altitude 
standard-rate  turns,  VOR  radial  interception  and 
tracking,  and  a  simple  positive-control  scenario  that 
incorporated  all  previous  components.  The  subse¬ 
quent  data  collection  consisted  of  more  challenging 
VOR  navigation  courses  and  compliance  with  ATC- 
issued  vectors  and  altitude  changes  as  well  as  an  ILS 
approach. 

Pilot  subjective  reports  were  collected  using  posttest 
questionnaires.  Comments  and  ratings  concerning 
handling  qualities  and  workload  consistently  indi¬ 
cated  that  pilots  judged  the  A-36  simulation  to  be 
more  sensitive  to  control  inputs  and  more  difficult  to 
fly  than  the  aircraft.  These  assessments,  however, 
appeared  to  correlate  with  piloting  style  (i.e.,  inter¬ 
ventionist  versus  noninterventionist).  Ratings  on  the 
flying  qualities  of  the  Sundowner  were  also  to  the 
“sensitive”  side,  but  not  to  the  degree  found  for  the 
Bonanza. 


Pilots  generally  reported  that  the  experimental  sce¬ 
narios  were  more  challenging  than  their  usual  flying 
and  thus,  presented  a  significant  workload.  This,  of 
course,  was  a  positive  bit  of  news,  as  the  intent  was  to 
load  the  pilots  sufficiently  to  detect  performance 
decrements.  Obtained  ratings  also  indicated  that  par¬ 
ticipants  felt  the  simulation  reasonably  represented 
flight  tasks  in  the  ATC  environment.  Ratings  were  all 
high  on  the  “task  realism”  scale.  A  number  of  indi¬ 
viduals  expressed  the  desire  to  spend  additional  time 
in  the  simulator  and  several  instructors  indicated  their 
interest  in  using  such  a  system  for  training.  Behaviors 
observed  during  the  flights  also  indicated  that  the 
participants  were  ego  involved  in  the  process  and  were 
responding  to  the  simulated  flight  much  as  one  would 
expect  them  to  respond  during  an  actual  flight.  This 
was  true  of  continuous  aircraft  control,  radio  naviga¬ 
tion,  and  communications  with  ATC. 

Continuous  Real-time  Performance  Data 

Flight  data  were  sampled  and  stored  at  0.5  Hz. 
Variables  recorded  included  latitude,  longitude,  mag¬ 
netic  variation,  altitude,  airspeed,  heading,  cross¬ 
track  error,  DME  indication,  and  a  number  of  status 
variables  (event  mark,  gear,  flaps,  marker  beacons, 
etc,).  Examination  of  the  data  suggests  that  perfor¬ 
mance  variables  that  can  be  sampled  at  lower  frequen¬ 
cies  (2  Hz  or  less)  and  that  represent  outcome  states 
(i.e.,  location  of  the  aircraft  in  three  dimensions 
relative  to  desired  altitude  and  ground  track)  can  be 
effectively  monitored  with  the  system  and  provide 
adequate  measures  of  aircraft  system  performance  for 
navigation  and  altitude  maintenance  tasks.  Examples 
are  shown  in  Figures  3 A  &  B  where  the  plan  views  are 
shown  for  one  pilot  to  compare  the  HSI  course  track¬ 
ing  with  the  VOR/DG  course  tracking.  It  is  evident 
that  better  acquisition  and  tracking  performance  was 
obtained  using  the  HSI,  consistent  with  the  proce¬ 
dural  data  previously  mentioned  and  with  previous 
studies  of  integrated  navigational  displays. 

Higher  sampling  frequencies  up  to  at  least  16  Hz 
are  possible  with  the  system,  but  storage  becomes  a 
problem  as  variables  are  held  in  memory  until  the  end 
of  a  flight.  The  present  system  could  be  suitable  for 
control-theory  studies  of  manual  flight  control  where 
the  expected  frequency  of  control  reversal  activity  is 
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Figure  3.  Example  flight  paths  for  one  subject  for  (A)  VOR/DG  and  (B)  HSI 
instrumentation.  Dotted  lines  indicate  desired  tracks. 


not  likely  to  exceed  the  measurement  capabilities  of 
the  system  during  most  normal  realms  of  flight.  The 
limitation,  as  suggested,  is  more  one  of  available  data 
storage  space,  limiting  the  duration  of  high-frequency 
data  collection.  Flight  attitude  data  are  presently 
being  transmitted  from  the  program  at  16  Hz,  so 
evaluation  of  maneuvers  based  upon  sampling  of 
aircraft  attitude  can  apparently  be  accommodated, 
although  such  were  not  evaluated  in  the  present  study. 
A  revision  of  the  software  is  presently  under  way  that 
would  allow  flight  data  to  be  transmitted  to  another 
computer  system,  thereby  alleviating  the  storage  space 
restriction  and  increasing  the  recordable  flight  dura¬ 
tions  even  at  high  sampling  rates. 

Observed  Procedural  Errors 

Two  categories  of  procedural/discrete  errors  were 
expected  to  be  observed  related  either  to  the  naviga¬ 
tion/orientation  problem  or  those  related  to  memory 
of  heading  and  altitudes  or  elements  of  the  verbally 
issued  clearance  amendments.  Navigation/orienta¬ 
tion  errors  included  inappropriate  setting  of  the  omni 
bearing  selector  (OBS),  flying  through  radials  with¬ 
out  any  corrective  action,  and  turning  in  the  wrong 
direction  for  an  intercept.  Memory  errors  included 
callbacks  for  heading,  altitude,  or  radial,  and  failure  to 
recall  present  assigned  altitude.  Errors  observed  for 
the  experienced  pilot  sample  are  summarized  in  Table  1 . 


These  findings  were  largely  as  anticipated,  demon¬ 
strating  more  errors  with  the  VOR/DG  combination 
and  without  bugs  than  with  the  HSI  and  bugs.  The 
direction  and  frequency  are  largely  consistent  with 
other  research  in  the  display  aiding/predigestion  lit¬ 
erature.  This  suggests  that  the  simulation  system  can 
be  useful  for  examining  problems  of  this  nature  where 
procedural  compliance  and  navigational  decision¬ 
making  are  involved  and  information  is  being  derived 
from  a  dynamic  instrument  representation. 

Reliability  and  Ease  of  Use 

The  system,  as  of  this  writing,  had  been  operated 
for  over  880  hours  (over  the  course  of  18  months). 
During  this  time,  only  two  failures  occurred  that 
required  the  halting  of  data  collection.  Both  of  these 
were  microprocessor-system  related,  one  being  a  disk 
controller  failure.  One  transient  failure  of  the  GE 
projection  system  was  observed,  but  it  did  not  inter¬ 
fere  with  data  collection.  No  failures  of  the  custom 
hardware  were  observed.  The  audio  system  amplifier 
experienced  one  failure  but  was  immediately  replaced 
by  on-site  available  equipment,  as  was  the  case  with 
other  components.  Maintenance  of  the  system  can 
generally  be  performed  on  site  by  individuals  with  a 
knowledge  of  personal  computer  systems.  The  system 
is  very  easy  to  use,  as  all  programs  are  run  automati¬ 
cally  following  initial  system  boot  using  batch  files 
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Table  1.  Procedural  errors,  11  subjects. 


VOR/DG  HSI 

Error  No  Bugs  Bugs  No  Bugs  Bugs 


VOR/Nav 

17 

12 

Memory 

5 

4 

Total  22  16  11 


and  all  operator-software  interfaces  are  straightfor¬ 
ward  and  easy  to  understand.  A  cold  start  of  the  system 
(from  power  off  to  full  operational  status)  requires 
approximately  one  minute. 

CONCLUSIONS 

The  data  obtained  to  this  point  indicate  that  the 
simulation  described  herein  has  sufficient  task  fidelity 
to  motivate  generalizable  behavior,  producing  out¬ 
comes  that  are  comparable  to  those  obtained  in  other 
simulation  devices  and,  in  fact,  aircraft.  The  ability  of 
the  system  to  collect  adequate  continuous,  real-time 
performance  data  has  been  demonstrated  with  refer¬ 
ence  to  tasks  limited  to  maintaining  aircraft  altitude 
and  track.  Control  theoretic  studies  are  likely  to 
require  some  modifications  of  the  software  to  provide 
sampling  rates  of  sufficiently  high  frequency.  It  was 
also  evident  that  the  task  environment  simulated  was 
sufficient  to  the  degree  that  behaviors/errors  likely  to 
be  observed  in  the  real  world  were  also  observable  in 
the  simulation,  allowing  for  the  examination  of  pro¬ 
cedural  error,  as  well  as  continuous  performance.  The 
simulation  has  been  comparatively  inexpensive  to 
integrate  and  maintain,  and  has  had  a  very  high  level 
of  availability. 

The  preliminary  indications  are  that  this  modular 
microprocessor-based  flight  simulation  system  can  be 
a  useful  and  economical  tool  for  examining  experi¬ 
mental  questions  involving  general  aviation  pilotage. 
No  data  are  as  yet  available  for  the  use  of  this  tool  in 
investigating  tasks  that  are  primarily  VFR  in  nature. 


Software  (textured  visual  databases)  and  hardware 
(100  MHz  and  faster  processors  and  faster  bus  archi¬ 
tectures)  developments  are  becoming  available  that 
will  further  enhance  the  possibilities  for  research  on 
visually  guided  behaviors  beyond  geographical  orien¬ 
tation,  pattern  flying,  and  visually  guided  approaches. 
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APPENDIX  A 


Figure  1  A.  CAMI  BGARS  schematic  diagram,  6-processor  configuration. 
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